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CheckMATE is a framework that allows the user to conveniently test simulated BSM physics 
events against current LHC data in order to derive exclusion limits. For this purpose, the data 
runs through a detector simulation and is then processed by a user chosen selection of experimental 
analyses. These analyses are all defined by signal regions that can be compared to the experimental 
data with a multitude of statistical tools. 

Due to the large and continuously growing number of experimental analyses available, users may 
quickly find themselves in the situation that the study they are particularly interested in has not 
(yet) been implemented officially into the CheckMATE framework. However, the code includes a 
rather simple framework to allow users to add new analyses on their own. This document serves as 
a guide to this. 

In addition, CheckMATE serves as a powerful tool for testing and implementing new search 
strategies. To aid this process, many tools are included to allow a rapid prototyping of new analyses. 
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PROGRAM SUMMARY 

Program Title: CheckMATE, AnalysisManager 
Journal Reference: 

Catalogue identifier: 

Licensing provisions: none 
Programming language: C++, Python 
Computer: PC, Mac 
Operating system: Linux, Mac OS 

Keywords: Analysis, Confidence Limits, Monte Carlo, Detector Simulation, Delphes, ROOT, LHC 
Classification: 11.9 

External routines/libraries: ROOT, Python 
Subprograms used: Delphes 

Nature of problem: The LHC has delivered a wealth of new data that is now being analysed. Both ATLAS and CMS have 
performed many searches for new physics that theorists are eager to test their model against. However, tuning the detector 
simulations, understanding the particular analysis details and interpreting the results can be a tedious and repetitive task. 
Furthermore, new analyses are being constantly published by the experiments and might be not yet included in the official 
CheckMATE distribution. 

Solution method: The AnalysisManager within the CheckMATE framework allows the user to easily include new experimental 
analyses as they are published by the collaborations. Furthermore, completely novel analyses can be designed and added by 
the user in order to test models at higher center-of-mass energy and/or luminosity. 

Restrictions: Only a subset of available experimental results have been implemented. 

Running time: The running time scales about linearly with the number of input events provided by the user. The detector 
simulation / analysis of 20000 events needs about 50s / Is for a single core calculation on an Intel Core i5-3470 with 3.2 GHz 
and 8 GB RAM. 
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IMPORTANT NOTE 

• CheckMATE is built upon the tools and hard work of many people. If CheckMATE is used in your publication it 
is extremely important that all of the following citations are included, 

— Delphes 3 [T]. 

— FastJet [a [3]. 

— Anti-fct jet algorithm |4]. 

— CLg prescription [a. 

— In analyses that use the Mx 2 family of kinematical discriminants we use the Oxbridge Kinetics Library 
Ii[7] and the algorithm developed by Cheng and Han [5] which also includes the variable 

— In analyses that use the Mqt family of kinematical discriminants we use MctLib [iniiii] which also includes 
the Mct± and McTy variables [12] ■ 

— All experimental analyses that were used to set limits in the study. 

— The Monte Carlo event generator that was used. 


I. INTRODUCTION 

CheckMATE m is a tool that allows for the easy testing of new physics models against current LHC data. The fully 
validated version now contains 14 LHC analyses with over 100 separate signal regions. More analyses are being added 
all the time and the beta-version now contains over 30 finished analyses that have been validated over significant 
regions of the parameter space. Including these analyses means that CheckMATE contains over 300 separate signal 
regions and this list is growing continuously. 

The tool is growing in popularity and many studies have now used CheckMATE to constrain both supersymmetry 
(SUSY) (e.g. [TllH6j ) and other models of new physics (e.g. HZIIH]) that may be probed at the LHC. In addition, 
the tool can be used to fit excesses that could potentially be the first signs of new physics (e.g. HSj). 

However CheckMATE is not only a tool to be used to re-interpret analyses that are already included in the program, 
it is also a powerful analysis environment for users to include other analyses or invent their own searches [20j . To this 
end, CheckMATE contains an AnalysisManager which makes the process of writing, testing and validating an analysis 
as easy as possible. In particular, the manager guides the user through the various choices of available final state 
objects, automatically writes an analysis template and calculates the various statistical measures required to define 
the sensitivity of a search. Many steps along this path are also significantly smoothed by extra features included in 
the CheckMATE package. 

Whilst similar analysis tools already exist, CheckMATE is specifically designed for new physics searches at the LHC. 
The analysis framework is heavily influenced by the Rivet |21j program and shares the philosophy of using projections 
to define the particular final state objects of interest. However, the big difference is that whilst Rivet is designed to 
recast unfolded high energy analyses, CheckMATE includes the detector simulation Delphes [T] with a significantly 
improved ATLAS tuningjj Consequently, CheckMATE can reproduce new physics searches where a detector simulation 
is vital to accurately reproduce the experimental result. Additionally, the integrated statistics functionality allows 
exclusion and/or discovery regions to be easily found with minimal user input. 

The MadAnalysis pMM] framework is another similar approach that also has an emphasis on new physics searches 
at the LHC. Since MadAnalysis also uses the Delphes detector simulation, work has now begun to improve the 
compatibility with CheckMATE so that the tools can be used interchangeably. 

For budding LHC phenomenology analysis authors, the most attractive feature is the simplification CheckMATE 
achieves in the analysis writing procedure. Many different tunes of final state objects are available (e.g. ‘loose’, 
‘medium’ and ‘tight’ electrons in case of ATLAS) while some objects even have the full multi-variate discriminant 
parametrised (e.g. &-jets) that automatically recalculate the fake rate depending on the efficiency required. Routines 
also exist for the automatic calculation of isolation variables where the user only has to specify the required parameters. 
Furthermore, many of the kinematical variables (e.g. tot 2 , ctT, razor) used by the LHC collaborations are included. 
Additionally, since Root is fully integrated, the TLorentzVector class is available which allows the easy access and 
calculation of a huge number of 4-vector variables. 


The implementation of an improved CMS tuning is planned in the near future. 
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To help with creating, testing and validating the analyses, tools are included for the automatic calculation of 
outflows and signal regions. The statistical evaluations required to assess the analysis are also implemented to assist 
the user at every point. Finally, if backgrounds and the corresponding errors are known, the experimental reach of 
the analysis to a particular model of new physics is also then calculated. 

In this manual, we will guide the users through the list of steps necessary to add these skeletons to CheckMATE, 
how these skeletons have to be understood and what has to be done in order to fill them with proper content. In 
Section [n] we introduce the main features of CheckMATE. Section m describes in detail the AnalysisManager module 
of CheckMATE and presents the main ingredients of analyses within CheckMATE. In Section |TV] we provide a detailed 
discussion of two analyses implemented in CheckMATE in order to familiarise the user with the analysis writing 
technique. This is followed by an example of a completely new analysis to test the discovery potential of a compressed 
SUSY scenario at the 14 TeV LHC. Finally we summarise is Section [V] 

II. OVERVIEW: THE ANALYSISMANAGER WITHIN CHECKMATE 

In this section we briefly summarize CheckMATE’s main program flow and outline the procedure to add a new 
analysis to the existing framework. Detailed information on CheckMATE itself can be found in m- 

CheckMATE uses simulated event files and cross sections as input. Users have to provide these and list the analyses 
they want them to be tested against. The provided files are then fully automatically processed by the following 3-step 
procedure: 

1. The embedded program Delphes takes the input events and applies a fast detector simulation. CheckMATE 
determines the settings required for Delphes based on the exact analyses chosen. This optimises the detector 
simulation by only requiring the hnal states objects needed for the particular analysis list. 

2. The output of the simulation is quantified by applying a standalone C-|—I- analysis program on the results. The 
final state objects of each event are tested to see if they fulhl the criteria of one or more of the signal regions 
defined within the analyses. For each of these signal regions, the total number of input events that pass the 
selection is evaluated and normalised to the given input cross section and luminosity. 

3. The predictions for all signal regions are compared to the stored experimental results in terms of the observed 
and expected number of events. This set of information is statistically combined to quantify the compatibility 
of the observation with the input model. CheckMATE then deduces whether the input model can be excluded 
or not to the 95 % C.L. 

In order to add a new analysis to this framework, CheckMATE requires various pieces of information and needs them 
stored in a well defined form at specific places in the code. The AnalysisManager offers a simple way to provide 
this information automatically to CheckMATE. It is already included in the official CheckMATE distribution and only 
has to be compiled explicitly, see Section [TV] Users can simply use terminal commands to list, add, edit or remove 
analyses. In order to add an analysis, a sequence of questions has to be answered which collects the information that 
CheckMATE requires to setup the internal workings accordingly: 

• Some general information (name of the author, name of the analysis, luminosity, -^/i, ...) is gathered first to 
define the headers and names of all the respective hies and to determine the normalisation of the event numbers. 

• The properties of hnal state objects determined on the detector level (jet clustering algorithm, working point 
efficiencies for jet havour tagging, lepton isolation criteria, ...) will later dehne the Delphes setup during the 
actual CheckMATE run as well as the available objects in the analysis code. 

• Numbers (observed number of events. Standard Model expectation, ...) and names for all signal regions have to 
be dehned by the user and are used in CheckMATE’s evaluation step. The user normally can take these directly 
from the corresponding experimental publication. 

After this information is entered, the AnalysisManager will create all the internal hies necessary to run the analysis 
within the normal CheckMATE framework. The user still has to dehne the actual analysis prescription in the form 
of a C-l—I- code. For that purpose, a skeleton source code is provided which gives the user automatic access to all 
objects dehned during the AnalysisManager step. Lots of additional predehned helper functions further reduce the 
effort to implement the analysis code. The call of a single function, countSignalEvent (<SR>), is sufficient to trigger 
the CheckMATE evaluation routines. In the upcoming sections we will discuss the dehnitions and features of all these 
objects and functions in more detail. 
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III. FEATURES OF CHECKMATE ANALYSISMANAGER 

As explained in the introduction, CheckMATE contains many novel features to make recasting or designing a new 
LHC analysis as simple as possible. In this section we introduce these features in detail while later sections give 
examples of how these are used in practice. 


A. Final state objects and functions 

Most of the reconstruction efficiencies for hnal state objects available in CheckMATE have been re-tuned according 
to ATLAS performance studies. In addition, new functionality has been included to allow a full reproduction of 
all the final state cuts currently implemented by the LHC experiments. Details of all the individual reconstruction 
efficiencies can be found in [T5] . 


Electrons 

All three of the current ATLAS electron definitions, loose, medium and tight, are available in CheckMATE and can 
be called in any analysis. They are different in terms of the reconstruction efficiency and these are implemented in 
terms of 2-dimensional look-up tables as a function of px and rj. A default loose isolation condition that requires the 
sum of calorimeter objects in a cone size of AR — 0.2 not associated to the electron to be less than 0.2 of the electron 
Pt is always applied. If tighter and/or more complicated isolation conditions are required please see the specific 
isolation subsection below. Also, in the case of ATLAS, a retuned algorithm is present to smear the reconstructed 
electron momentum. For CMS, the default Delphes tunings for electrons are used. 


Muons 

The majority of ATLAS analyses currently use one of two muon reconstruction algorithms, ‘Combined’ (named 
‘muonCombined’ in CheckMATE) and ‘Segment-Tagged’. The latter is usually used in combination with the first 
to maximise the muon efficiency and is hence called ‘Combined-I-Segment-Tagged’ (named ‘muonsCombinedPlus’ 
in CheckMATE). Both definitions, ‘muonsCombined’ and ‘muonsCombinedPlus’, are implemented in CheckMATE as 
2-dimensional look-up tables as a function of rj and cj). Again, in the case of ATLAS, a retuned algorithm is present 
that accurately smears the reconstructed muon momentum. As for electrons, the default Delphes CMS tuning for 
muons is used. 


Jets 

Jets are reconstructed using either smeared calorimeter deposits in the case of ATLAS or particle flow information 
in the case of CMS. By default Q the anti-ZcT algorithm implemented in FastJet [HE] is chosen to cluster jets and the 
user only has to specify the cone size (Ai?) required. Further studies of jet-substructure are possible since the user 
has access to all jet-constituents, tracks and calorimeter towers. 


Taus 

The tau reconstruction algorithm searches the event tree for the presence of a tau and, if the momentum vector 
determined from its visible decay products overlaps with a reconstructed jet, identifies the jet as a tau candidate. If 
not, it is considered a fake background candidate, for which different efficiencies are used. These jets are then further 
sub-divided into two categories, 1-prong or multi-prong depending on whether the jet contains exactly one or more 
reconstructed tracks. For each category CheckMATE has three different reconstruction algorithms (loose, medium and 
tight) parametrised as a function of px and t] that the user can easily select. Currently only ATLAS parametrisations 


^ The default setting can be changed in write_delphes_niodules .py module, under the key JetAlgorithm. See the Delphes manual for 
more details [I]. 
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are available with CMS functions planned for a future release. For CMS analyses, the ATLAS parametrisations are 
used. It should be noted that the tau parametrisation is specifically tuned to work with the anti-fey jet algorithm and 
a cone size Ai?=0.4. Using different jet settings will effect the reconstruction efficiencies and are not recommended. 


B-jets 

The fo-jet reconstruction function begins by first defining whether the jet overlaps with a 6 -quark. If not, it is 
checked whether there is overlap with a c-quark and if again there is no overlap, it is assumed that it only contains 
light quarks. All three types of jet have different probabilities to be tagged as a final state 6 -jet depending on the pt 
and 77 of the jet. 

A feature of CheckMATE is that the complete receiver operator curve (ROC) of the ATLAS multivariate 6-tagger is 
implemented. Consequently the user can choose any particular 6-tag reconstruction probability that they desire and 
CheckMATE will automatically adjust the probability that c-quarks or light jets fake a true 6-quark. This feature is 
very useful since ATLAS uses widely different tagging points depending on the analysis of interest and the associated 
fake rates vary by orders of magnitude. Currently only ATLAS parametrisations are available but CMS functions 
are planned for a future release. For now, the tuned ATLAS parametrisations are used for CMS analyses with the 
assumption that for a given working point the deduced efficiency distributions are roughly equal. 


Tracks and Calorimeter Cells 

Although not routinely required for most analyses, all reconstructed tracks and calorimeter cells are available. This 
information can be used for example to develop more complicated isolation procedures than those already implemented 
(see Isolation below) or for custom jet algorithms. For the track reconstruction efficiency and momentum smearing we 
use the default Delphes parametrisation for both ATLAS and CMS. The same is also true for the calorimeter energy 
smearing. 


Missing Energy 

Missing energy is reconstructed from the sum of the smeared calorimeter deposits in the case of ATLAS or the 
sum of the smeared particle flow objects in the case of CMS. In addition, to effectively parametrise additional QCD 
activity due to pile-up we also include an extra smearing factor on the missing energy that has been tuned to match 
the ATLAS distributions, but is used for both ATLAS and CMS analyses: this smearing is done by adding a vector 
with uniformly random and direction magnitude which follows a Gaussian distribution with width 20 GeV centred 
around 0. Due to the way Delphes treats final state muons, their contribution has to be added separately to the 
missing energy, as will be shown in the following Section. 


Isolation 

CheckMATE supplies routines that automatically calculate the various isolation conditions required by the LHC 
experiments for both muons and electrons. Firstly the user can specify whether the isolation is calculated with 
respect to the calorimeter or to reconstructed tracks. In addition, the option to calculate isolation as an absolute pt 
or as a fraction of the reconstructed lepton px is available. For finer control, the thresholds that individual tracks or 
calorimeter cells are included in the isolation calculations can also be stated. 

Whilst the above options allow reproduction of the vast majority of isolation variables used by the experiments we 
also remind the user that both the tracking and calorimeter information is available if more control is required. 


Final state projections 

In order to simplify the basic acceptance cuts required on final state objects, CheckMATE employs a projection 
based system that is very similar to the one pioneered by Rivet [2T]. The idea is that a vector of final state objects is 
given to a function along with the the angular acceptance and minimum pp. A vector of the objects that passed the 
cuts is returned and this avoids multiple loops over final state objects being needed in the analysis code. In addition. 
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ATLAS often performs a cut on final state objects that fall into the transition region between the central and forward 
detectors and an optional projection for this area is included. 


Overlap removal 

Another commonly required routine for LHC analyses is the removal of overlapping final state objects. For example, 
since electrons deposit energy in the calorimeter they are also included in the jet algorithm and must be removed from 
the list of jets in order not to double count momentum. In addition many analyses also require an overlap removal 
between muons and jets as well. CheckMATE allows easy overlap removal where the user only has to state the cone 
size required for the operation, the object to be removed and the object to be kept. 


B. Kinematical Variables 


CheckMATE has access to both the Delphes kinematical variables and the TLorentzVector class from Root. The 
Delphe^ library contains many of the most used kinematical parameters, like transverse momentum, energy and 
pseudo-rapidity, as well as non-kinematical properties, for example electric charge. In addition the TLorentzVectoi]^ 
class contains the full list of Lorentz variables and the possibility to calculate values from two separate vectors, e.g. 
relative angles. 

CheckMATE also contains a number of popular LHC kinematical variables which are either hard-coded or sourced 
from other libraries (Oxbridge Kinetics Library [BH5] and MctLib [IHIIII])- For a more detailed review of many of 
variables that have now been developed for high energy physics please see [5S]. 

• Mt [M®: 

The transverse mass. My, allows for the reconstruction of a particle that decays semi-invisibly via an end-point 
in the kinematical distribution. We define, 

= "ifnvis + - COS Ac/)) , (I) 

where is the magnitude of the transverse momentum of the visible final state particle which is assumed to be 
massless. Winvis is the mass and is the magnitude of the missing transverse energy of the invisible particle 
and A(j) is the azimuthal angle between the visible final state particle and the missing energy vector. 

Mp is most commonly used to identify and measure the leptonic decay modes of the W boson, W ^ iv in 
which case both of the final state particles, lepton and neutrino, can be considered massless. 

• Mx2 [Z1 [H] ■ 

In the case that an event contains two or more missing particles, an end-point in Mp to measure the decaying 
particle mass will no longer exist. This is true when we for example consider the production of two particles 
that both decay to a final state that contains an invisible particle. In addition, for new physics searches we are 
also often faced with the possibility that the masses of the invisible particles are unknown. If however the two 
invisible particles have the same mass, the space of possible momentum configurations is reduced and in fact 
can be numerically tested. To account for these scenarios, the Mt 2 variable was defined [7] which minimises 
over all possible partitions and ^he vector missing energy between the two decay chains. 


MT2{m^) 


min 




[max{MT(pf,^f;mx), Mt(p^, m^)}]. 


( 2 ) 


Here, P^ 2 ) vector transverse momentum of the visible final state particles, assumed to be massless, and 

is the mass of the invisible particle, which may be unknown and set to some test value. 

• A7I4 0: 

A popular variable to efficiently discriminate scalar top partners which decay into top quarks and an invisible 
particle from the SM top background at the LHC is This is a modification of standard Mt 2 to an 


^ A full list of the available functions is given at https://cp3.irinp.ucl.ac.be/projects/delphes/wiki/WorkBook/RootTreeDescription 
A full list of the available functions is given at https://root.cern.ch/root/html/TLorentzVector.htinl 




asymmetric case where a single lepton and two 6-j6ts are reconstructed. The aim for M ^2 is to reconstruct the 
top-quark mass in di-leptonic top decays where one lepton fails to be reconstructed. Thus, the decay chain with 
a reconstructed lepton should evaluate with a massless neutrino, while the other chain — assumed to 
be missing a lepton — should reconstruct the W mass, mw, 

min [max{MT(p^^-f . (3) 

1^1 =T 

Here, i® vector transverse momentum of the &-jets and pj is the vector transverse momentum of the 

lepton. Commonly, both combinations of the &-jet momentum with the lepton momentum are computed and 
the smallest resulting is used. 

Mct |10j : 

The Mt variable (Eq. [H is invariant under Lorentz-boosts. More specifically, it is invariant under co-linear 
transverse boosts, i.e. ifboth particles are boosted by the same magnitude in the same transverse direction. 

A related observable called Mct, 


M^t = ml+ml + 2 {E ^EJ -b pf • p^), 


(4) 


can be defined, where the transverse energy is defined as, Ej = \/m1 + |pf p. Contrarily to Mt, it is invariant 
under contra-linear transverse boosts , i.e. under boosts with equal magnitude but opposite direction in the 
transverse plane of the two final state particles. This is useful at a hadron collider since in the absence of initial 
state radiation (ISR), the production of a pair of equal mass particles will have equal back to back transverse 
boosts. Moreover, these boosts will be unknown if the parent particle decays to an invisible particle. 


Mct corrected m- 

A drawback of Mct is that the variable is not invariant under transverse boosts of the laboratory centre-of-mass 
frame. Consequently, transverse boosts from ISR will cause an end-point measured from Mct to be smeared. 
However, a procedure to correct for such boosts was detailed in m and is implemented via MctLib. 


Mct± and Mct^ |12| : 

In event topologies that allow for the reliable determination of ISR, two one dimensional decompositions of 
Mct can be made. We first define the decomposition of the final state momentum vectors into the components, 
perpendicular and parallel, to the ISR transverse vector, U^, 


= pf - pf" = 


X (pf X U^). 


(5) 

( 6 ) 


We can now define the one dimensional analogues of Mct, 

Mct^ = ml + ml+ 2 (aJ" + pf" • pf"), (7) 

Mct, =ml + ml + 2 (<'' <" + pf" • pf'), (8) 


where we also dehne the corresponding energy decompositions, 



(9) 


The perpendicular variable, Mct±_ , is calculated using vectors perpendicular to the ISR and thus is invariant 
under boosts in the direction of ISR. 


OiT [311132] : 

A substantial background to missing energy searches with jet final states are pure QCD events where one jet 
has a large mismeasurement. The mismeasurement can lead to a significant missing energy signal that could 
fake new physics with genuine missing energy and the variable ut, was designed to guard against this. For a 
di-jet system the variable is defined as, 
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where Ej^ is the less energetic of the two jets and Mxj the transverse mass of the di-jet system, 


Mtj - 



( 11 ) 

( 12 ) 


For a perfectly back to back di-jet system, ax = 0.5, but if one of the jets suffers a mismeasurement in energy 
the resulting ax will be lower. In contrast, signal like events that do not have a back to back topology but 
genuine missing energy can have values of ax >0.5. 

The variable is also generalised to multi-jet topologies, 


where. 


Hx — /^Hx 
ax = . 

- $x 


Ht = ^E 



(13) 


(14) 

(15) 


AHx is found by forming two pseudo jets from all the reconstructed jets in the event. The combination is done 
by performing a scalar sum of the jet Ex and choosing the jet combination that minimises the absolute Ex 
difference (AHx) between the two. 


• Razor [55H55] : 

Another approach that has been designed to focus on the pair production of new states, e.g. scalar quark 
partners, that each decay into a visible and an invisible particle, e.g. a neutralino and a jet, has been called 
‘razor’. Two dimensionful variables are defined. 


Mb. = \J (|pi| + |P2|)2 - {pI+pIY , 


M^' = 




(16) 

(17) 


where Mb is designed to peak at the mass scale of the new physics and measures the transverse momentum 
imbalance. Combining the two values in a ratio. 


R = 




M, 


R 


(18) 


one finds a variable that is expected to peak at i? ~ 0.5 for two body decays. In the Standard Model, both Mb 
and R are expected to fall smoothly and thus new physics can appear as a bump on a falling background. 

As for ax, the variable can be generalised to topologies that contain more than two visible final state particles 
by first performing a clustering. In this case, two ‘megajets’ are formed by summing the four-momentum of all 
combinations of final state jets and leptons. The combination that minimises the sum of the invariant masses 
of the two megajets is then chosen. 


C. Validation 

An extremely important step on the way to recasting an LHC analysis is validation in order to ensure that all 
kinematical cuts are implemented correctly. LHC analyses now routinely provide information for particular hypothet¬ 
ical signal models concerning the acceptanc^ as each individual cut is applied in order. This information is vitally 
important when validating an analysis since it allows the acceptance of each individual cut to be checked. 


® Depending on the analysis, this information can also be in the form of cross sections or simply Monte-Carlo event counts. 
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In order to make the validation procedure as simple as possible, CheckMATE includes a specific syntax for recording 
outflows that can then be easily checked against the experimental publication. The syntax simply replaces the call 
countSignalEvent (<SR>) with countCutFlowEvent(<CF>) and generates completely separate outflow results that 
can be quickly modihed and do not interfere with the defined signal regions. 


D. Evaluation statistics 

One of the most convenient features of CheckMATE is the evaluation of the number of signal events and the 
corresponding calculation of all of the statistics required to determine if a particular model is excluded at the LHC 
or not. In fact the easiest way of running CheckMATE simply leads to the result excluded or allowed depending on 
the number of signal events. More results and statistical variables are available for more advanced users. All of the 
above procedures are automatically evaluated for newly defined analyses with minimal user input required. 

For the implementation of existing LHC analyses, the user simply has to provide the number of background events 
expected in a particular signal region, the associated error and the actual number of observed events. All the normal 
CheckMATE evaluation routines are then available automatically and work with both weighted and unweighted signal 
events. 

In the case that the user is developing a new analysis, the Standard Model backgrounds must first be evaluated 
before the expected reach at the LHC can be calculated. For this the user simply runs CheckMATE as normal on the 
various backgrounds that are expected to contribute. The number of events normalised to the cross section is then 
evaluated by CheckMATE process by process for each signal region and simply summed to give the total background. 
Once this procedure is complete the analysis can then be used as normal. 

In addition, since we inherit all libraries contained within Root, any of the statistical functions and histogramming 
features are easily accessible for use within analyses. 


IV. PRACTICAL EXAMPLES 

Often it is easiest to do a task oneself after one has seen it in a practical example. This is why in this section we 
will discuss the implementation of three practical examples in detail. 


A. Example 1: Recasting a simple analysis step by step 


In this part we will add an (already existing) analysis to the CheckMATE framework from the very first steps on. 
We assume that CheckMATE is already installed in the directory <CM>. 


Running the Analysis Manager 


If <CM>/bin/AnalysisManager does not yet exist, we run make AnalystsManager within <CM> to create it. Starting 
the AnalysisManager will open the introduction header and a first prompt: 


/ _I l__ _ _I I .1 \/ I / \l_ _l_I 

II I ’_ \ / _ \/ __l 1/ / l\/l I / _ \ I I I _l 

II _I I I I <11 11/ _ Mill _ 

\_l_l l_l\_l\_l_l\_\_l L/V \_\_l I_I 


/\ _ _ I _ l\/l ______ 

/—\l )(_ll\/_)l_) I I (_l I )(_!(_)(-! 

/ _/ 


What would you like to do? 

-(l)ist all analyses, 

-(a)dd a new analysis to CheckMATE, 

-(e)dit cinalysis information, 

-(r)emove an analysis from CheckMATE 

Apparently we want to add something new, so we hit a: 

This will collect all necessary information to create a full analysis and 

Tcikes care for the creation and implementation of the source files into the code. 

Please answer the following questions. 

Attention: Your input is NOT saved before you answered all questions! 
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The first block of questions gathers some general information on the analysis. At the beginning, you are asked for 
your name and your email address. These are printed on the very first lines of the analysis code to make sure that 
the right author can be contacted in case something goes wrong. 

1. General Information to build analysis 
Your Name (to declare the analysis author): 

Guybrush Threepwood 
Your Email: 

threepwoodSpirates.arr 


Afterwards, some simple information on the analysis has to be provided. The name should be short but cleaij^ and 
should not contain any spaces, since this string defines the names for all analysis-specific files. 

I Analysis Name: 

atlas_conf_2013_047X 


The one-line description is the one used when someone types 1 in the AnalysisManager. It can also be found in the file 
list_of_analyses.dat. The multiline description is printed on the top of every output file the respective analysis 
code creates: 

Description (short, one line): 

ATLAS, 0 leptons + 2-6 jets + etmiss 
Description (long, multiple lines, finish with empty line: 

ATLAS 

ATLAS-CONF-2013-047 
0 leptons, 2-6 jets, etmiss 
sqrt(s) = 8 TeV 
int(L) =20.3 fb“-l 


The luminosity is important for the correct normalisation of the analysis results: 

I Luminosity (in fb''-!) : 

I 20.3 

At the end of this block, the AnalysisManager will ask you whether you want to provide control regions for the given 
analysis. If we choose “yes”, it will produce a second analysis source file which can be run separately from within 
CheckMATE. In our case, we are only interested in implementing signal regions and outflows for which we do not need 
a second analysis file. 

I Do you plan to implement control regions to that analysis? [(y)es, (n)o) 

I 

Now we go more into the details of the analysis. As the next step, we have to list all signal regions the analysis 
provides. If we check out the actual analysis publication [36], we find five main signal regions A-E which sometimes 
are split into subregions L(oose), M(edium) or T(ight). We enter them one by one 

2. Information on Signal Regions 

List all signal regions (one per line, finish with an empty line): 

AL 

AM 

BM 

BT 

CM 

CT 

D 

EL 

EM 

ET 


In our case, we implement an experimental study for which the experimental observation and Standard Model expec¬ 
tation are known. If we want to create a new analysis, see later sections, we will not know these and have to find and 
enter them later. 

I Is the SM expectation B known? [(y)es, (n)o]? 

I y 


® Note that we only added the letter X at the end of the analysis name to prevent the AnalysisManager from overwriting the already 
implemented atlas_conf _2013_047 analysis data. 
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Now things become a little more involved: for each of the above signal regions, we have to provide a set of numbers 
to which CheckMATE can compare the model prediction. In its most simple form, this includes the observed number 
of events obs and the Standard Model expectation bkg including error bkg_err. Sometimes, the background error 
is split up into a statistical and systematical error and sometimes the systematical error is given in asymmetrical ± 
form. The different input options also allow for the data to be given in all these different formats. In our example, 
Table 4 in ref. [36] gives us the minimal set: obs, bkg and bkg_err. 


Information: We are now going to ask you which numbers you want to provide for each signal region. 

The following items are possible: 

obs: Observed number of events 

bkg: Expected number of background events 

bkg_err: Expected total error on bkg 

bkg_errp: Expected total upper error (in case of asymmetric errors) 

bkg_errm: Expected total lower error (in case of asymmetric errors) 

bkg_err_stat: Expected statistical error on bkg 

bkg_err_sys: Expected systematical error on bkg (in case of symmetric errors) 

bkg_errp_sys: Expected systematical upper error (in case of asymmetric errors) 
bkg_errm_sys: Expected systematical lower error (in case of asymmetric errors) 

Note that not all of these numbers have to be given (e.g. you don^t have to give the total error if you give the individual stat and 
sys contributions) 

However, there are some requirements, about which you will be warned if you don^t meet them (e.g. giving xyz_errp without xyz_errm) 
The standard, minimum set of information consists of obs, bkg and bkg_err 
List all categories you want to supply (one per line) 
obs 
bkg 

bkg_err 

The set of information you entered is valid. 


The AnalysisManager would reject the input list if it was not complete: for example, if we entered only the statistical 
error bkg_stat but no systematical error. 

Next, we have to enter the actual numbers for all the above categories for each and every one of the listed signal 
regions: 


You now have to add the numbers for each of the given signal regions. 

Note that while you enter more numbers, the corresponding model independent 
95“/o confidence limits for the items you have already entered are calculated 
in the background. 

AL 

obs: 

5333 

bkg: 

4700 
bkg_err: 

500 

S95obs cind S95exp values are calculated internally (progress: 0/2) 

AM 

obs: 

135 

bkg: 

122 
bkg_err: 

18 

S95obs cind S95exp values are calculated internally (progress: 0/4) 

[...] 

ET 

obs: 

5 

bkg: 

2.9 
bkg_err: 

1.8 

S95obs and S95exp values aire calculated internally (progress: 3 / 20) 


To allow for a fast statistical evaluation of the result in CheckMATE, it is convenient to first translate observation and 
expectation into a model independent upper limit on any new signal prediction S95. These numbers are calculated 
internally by the AnalysisManager. As this calculation takes some time, it is queued and calculated in the background 
while the user enters the detector related information. 

The next set of questions are all about detector level objects: 

3. Settings for Detector Simulation 
3.1: Miscellaneous 

To which experiment does the cinalysis correspond? [(A)TLAS, (C)MS] 

A 

As a next step we have to enter information regarding lepton isolation criteria. Normally, these are defined within 
the experimental publications and are needed to distinguish signal leptons from leptons within jets. In our example, 
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however, there are no signal leptons and therefore we do not require any particular isolation criteria for neither 
electrons, muons nor photons. 

3.2: Electron Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 
n 

3.3: Muon Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 
n 

3.4: Photon Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 
n 

Note that even though we entered no, internally there will always be a soft isolation condition that cannot be 
overwritten by the user and which is automatically applied on electrons, muons and photons. This ensures that 
objects that would not have been reconstructed due to overlapping detector activity are automatically removed 
internally. 

We now continue with the definition of jets. Reference [36] tells us the exact properties we have to enter here: 

3.5: Jets 

Which dR cone radius do you Wcint to use for the FastJet algorithm? 

0.4 

What is the minimum pt of a jet? [in GeV] 

20 

Do you need a separate, extra type of jet? [(y)es, (n)o] 
n 

The extra type of jet is needed in the rare case that a single analysis requires two different jet cone size, AR, however 
it is not required in our example. The final questions relate to potential flavour tags we have to apply on jets. The 
given analysis considers 5-tags with a reconstruction efficiency of 70%. We therefore simply enter 

Do you want to use b-tagging? [(y)es, (n)o] 

y 

b-Tagging 1: 

What is the signal efficiency to tag a b-jet? [in %] 

70 

Do you need more b tags? [(y)es, (n)o] 
n 

Do you want to use tau-tagging? [(y)es, (n)o] 
n 

Depending on how quickly we entered the above information, the AnalysisManger either finished the internally started 
S95 calculations or will wait until these are complete. When the evaluation ends, the results are shown and should 
be briefly checked to make sure that all numbers are sensible. 

All necessary information has been entered. Before the AnalysisManager 
can create all required files, the internal S95obs and S95exp 
calculations have to finish. The calculation should taike 10s up to a minute 
per point. 

... done! 

Please check the below results for sanity. If anything looks 
suspicious, please contact the CheckMATE authors, 
obs bkg bkgerr S95obs S95exp 
5333 4700 500 1400 985 

135 122 18 50 40 

29 33 7 14 17 

4 2.4 1.4 6.9 5.8 

228 210 40 87 79 

0 1.6 1.4 3.0 3.4 

18 15 5 15 13 

166 113 21 90 54 

41 30 8 28 21 

5 2.9 1.8 8.2 6.6 

(Press any key to continue) 

Note that due to numerical effects, your results might differ from the numbers above. Comparing the calculated 
numbers to the ones in Ref. [36] tells us that the numbers are acceptable. Usually, the experimental numbers 
differ from the ones calculated by the AnalysisManager. This is mainly caused by different parametrisations of the 
background uncertainties, taking information on individual error sources into account to which CheckMATE does not 
have access. CheckMATE still prefers to calculate and use its own numbers as the exact same statistical routines for 
S95 are used for the proper CLs calculation in the CheckMATE evaluation routines such that the results of the two 
approaches are more consistent with each other. 

We therefore accept the numbers and finish the AnalysisManager section: 
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- Variable values saved in /hdd/sandbox/managertest/data/atlas_conf_2013_047X_var.j 

- Created source file /hdd/sandbox/managertest/tools/analysis/src/atlas_conf_2013_047X.cc 

- Created header file /hdd/sandbox/nianagertest/tools/analysis/include/atlas_conf_2013_047X.h 

- Updated Makefile 

- Updated main source main.cc 

- Reference file created 

- List of analyses updated 

Analysis atlas_conf_2013_047X has been added successfully! 

Run ’autoreconf; ./configure {parameters}; maike’ to compile the new sources. 

You can find the list ’{parameters}’ you configured CheckMATE originally with at the beginning of the file ’config.log’. 


Looking at the Skeleton Code 

The AnalysisManager is mainly responsible to gather all the information for the detector simulation and the 
statistical evaluation sections of CheckMATE. What is still left to do is to define the analysis code that is used 
to analyse the events the user provides. In this section we describe how to do this by continuing our example of 
atlas_conf_2013_047X. Note that the code we develop here differs slightly from the actual code implemented in 
CheckMATE since we, for the sake of understanding, change the order of some steps here and shorten the code where 
possible. It still gives the exact same results for the signal region tests though. 

As the AnalysisManager has told us, source and header file for our analysis have already been created. These are 
already filled with skeleton code that properly compiles and embeds the analysis into the existing framework. What 
should concern us most is the source file which contains the actual analysis code. Every CheckMATE analysis contains 
three main functions 0 

initializeO: This function is called once at the beginning of an analysis. It is therefore used to setup overall 
information, initialise variables and open files that are used throughout the analysis. By default, AnalysisManager 
data that is important for the analysis is already embedded]^ 

#include "atlas_conf_2013_047X.h" 

// AUTHOR: Guybrush Threepwood 

// EMAIL: threepwoodOpirates.arr 

void Atlas_conf_2013_047x::initialize0 { 

setAnalysisName("atlas_conf_2013_047X"); 

setinf ormationC " 

"# ATLASXn" 

"# ATLAS-CDNF-2013-047\n" 

"# 0 leptons, 2-6 jets, etmiss\n" 

"# sqrt(s) = 8 TeV\n" 

"# int(L) =20.3 fb‘-l\n" 

setLuminosity(20.3*units::INVFB); 

ignore("towers"); 

ignore("tracks"); 

bookSignalRegions("AL;AM;BM;BT;CM;CT;D;EL;EM;ET;"); 

} 

The setAnalysisName and setinf ormation functions define human readable headers and the prefix of all analysis- 
related output files. setLuminosity is used to properly normalise the input to a physical number of events given the 
cross section the user provides in the CheckMATE input card. This number can be used to rescale the overall output, 
e.g. to take a global event-cleaning efficiency into account, ignore functions are for pure computing time optimisation 
reasons: they ignore information in the detector simulation to be read out if it is not required by the user. This is 
usually the case for calorimeter towers and tracker information which forms large datasets but which are rarely needed 
for the actual analysis. Finally book functions are defined for SignalRegions and Cutf lowRegions. These functions 
make sure that the respective _signal.dat and _cutflow.dat analysis output hies contains numbers for each of the 
booked signal/cuthow regions. No matter in which order the signal regions are booked they will always be sorted 
alphabetically in the output. 

analyze(): This function is the heart of any analysis: It is called once per event and contains the physics that is 
used to quantify the given data. Except for some comments and tips, this function does not contain any code yet and 
we will hll it in the next part of this tutorial. 

finalize 0: As the name suggests, this function is called once at the end of a given analysis. Usually this function 
is empty, however it can be used to e.g. free pointers that have been dehned during initialize or to close some extra 
hies that have been opened and hlled before. 

Now that we have understood the structure let us start to implement the analysis code. 
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The three-function structure is similar to the one used in the Rivet framework. 

Note that the actual source file includes many extra lines of explanatory comments which we have removed for this tutorial. 
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Selection of Final State Objects 

As the first step, we should define all the final state objects that we need and apply the appropriate kinematical 
cuts. The given analysis requires the following objects: 

• missing energy 

• electrons with px > 10 GeV and \r]\ < 2.47 that pass minimal selection criteria 

• muons with px > 10 GeV and |? 7 | < 2.4 reconstructed with combined data from tracker and muon spectrometer 

• photons with px > 130 GeV and |? 7 | < 2.47 excluding 1.37 < I 77 I < 1.52 and 

• jets with Px > 20 GeV and |? 7 | < 2.8. 

In the code, all the above objects are already predefined and we just have to select the right kinematic range using 
the f ilterPhaseSpace function of the analysis base class: 

void Atlas_conf_2013_047::analyzeO { 
missingET->addMuons(muonsCombined); 

electronsLoose = filterPhaseSpace(electronsLoose, 10., -2.47, 2.47); 
muonsCombined = filterPhaseSpaceCmuonsCombined, 10., -2.4, 2.4); 
jets = filterPhaseSpaceCjets, 20., -2.8, 2.8); 
photons = filterPhaseSpace(photons, 130., -2.47, 2.47, true); 

The missing energy vector does not a priori contain muon contributions as different analyses can have different 
definitions of the construction of the missing momentum vector. We therefore add the contribution of all reconstructed 
muons by using the addMuons function. Furthermore, the area 1.37 < \rj\ < 1.52 is often excluded for objects 
reconstructed in the electromagnetic calorimeter. The f ilterPhaseSpace function therefore has an optional last 
parameter which — if set to true — ignores objects in exactly that region. 

These objects are then tested against different overlap conditions. In our case, these are defined as follows: 

1. First, any jet with AR{jet, e) < 0.2 to a nearby electron is removed. 

2. Then, any electron and any muon with Ai?(£,jet) < 0.4 is removed. 

Functions that take care of these removals are fortunately already available in the AnalysisBase class: 

jets = overlapRemoval(jets, electronsLoose, 0.2); 
electronsLoose = overlapRemoval(electronsLoose, jets, 0.4); 
muonsCombined = overlapRemoval(muonsCombined, jets, 0.4); 

Beware that the order of these steps is crucial. Due to the setup of the detector simulation most electrons will also 
be reconstructed as jets. An overlap removal with respect to these two objects is therefore always necessary to avoid 
severe double counting issues. 


Event Selection 


Now that we have finished the object reconstruction step, we can start checking whether a given event fulfils the 
criteria of the given analysis. In our example, we should ignore events with leptons and hard photons. This is done 
very easily by using the return statement which ends the current run of the analyze!) function and hence effectively 
vetoes the currently processed event: 

I if(’photons.emptyO || !electronsLoose.empty() || !muonsCombined.empty()) 

I return; 

We now start to look at the signal region criteria which are summarised in Table I in [36]. Firstly all signal regions 
require 160 GeV and at least two reconstructed jets. From these the leading jet must have a px of least 130 
GeV and the sub-leading jet px > 60 GeV. All objects in GheckMATE analyses have a PT member that can be directly 
accessed for this purpose: 

if(missingET->PT < 160.0) 
return; 

if (jets. sizeO < 2 || jets[0]->PT < 130 || jets[l]->PT < 60) 
return; 





16 


Note that all object vectors are automatically sorted with respect to px such that leading and sub-leading jet are 
simply the first and the second object in the jet vector 

Furthermore the angular separation of the missing momentum vector and the leading two jets must be at least 0.4. 
We make use of the P4() function of each object which returns the corresponding TLorentzVector object defined in 
ROOT. This class comes with a large list of procedures to calculate 4-momentum parameters like the A(j) separation 
we need: 

if (fabs(jets [0]->P4().DeltaPhi(missingET->P4())) < 0.4) 
return; 

if( fabs(jets [1]->P4().DeltaPhi(missingET->P4())) < 0.4) 
return; 


There are more constraints on further ‘hard jets’ with px > 40 GeV: If a third ‘hard jet’ exists, it has to pass the 
above criterion too. For some signal regions, an additional constraint is applied if extra jets appear in the event. Here 
the looser constraint that Acj) > 0.2 is applied to the fourth jet onwards: 

std::vector<Jet*> hardjets = filterPhaseSpace(jets, 40., -2.8, 2.8); 

bool validThirdJet = (hardjets.size() < 3 || fabs(hardjets[2]->P4().DeltaPhi(missingET->P4())) > 0.4); 

bool validMultiJet = validThirdJet; 

for (int j = 3; j < hardjets.size(); j++) { 

if (f abs (jets [j]->P4() .DeltaPhi (inissingET->P4())) < 0.2) 
validMultiJet = false; 

> 

For our selection we need the ratio of to the total effective mass meff(fVj) as well as to y/Hr- Ht is defined to 
be the scalar px sum of of all jets with px > 40 GeV whereas meg{Nj) is the scalar px sum of the leading N jets 
plus .^x- There is also a requirement on meff(incl) := Ht+^t- We have to calculate these explicitly for all allowed 
number of jetsp^ 

double HT = 0.; 

for(int j = 0; j < hardjets.size(); j++) 

HT += hardjets[j]->PT; 
double mEffincl = HT + missingET->PT; 

double mEff2 = missingET->PT + jets[0]->PT + jets[l]->PT; 
double rEff2 = missingET->PT/raEff2; 

double mEff3 = 0; 
if (jets.sizeO >= 3) 

mEff3 = mEff2 + jets[2]->PT; 
double rEff3 = 0; 
if (jets.sizeO >= 3) 

rEff3 = missingET->PT/mEff3; 

double mEff4 = 0; 
if (jets.sizeO >= 4) 

mEff4 = mEff3 + jets[3]->PT; 
double rEff4 = 0; 
if (jets.sizeO >= 4) 

rEff4 = missingET->PT/mEff4; 

[...] 

double rEffHT = missingET->PT/sqrt(HT); 


The last requirement we have to check is that the momentum of the leading N jets in a given signal region is larger 
than 60 GeV. For that we simply count the number of jets after cutting to the signal phase space: 

I int nSignalJets = filterPhaseSpace(jets, 60., -2.8, 2.8).size(); 

With all these numbers at hand we can go ahead and check the various signal regions. Whenever an event passes the 
respective criteria, we just have to call the countSignalEvent function: 


^ Standard C-l—h vectors do not catch out-of-bound indices. Users therefore have to ensure semantically that jetsfi] is only accessed if 
jets contains at least i-\- 1 members. 

The authors are aware that the conditions could be formulated more concisely by using the ternary ? operator, e.g. 

double inEff4 = jets.sizeO >= 4 ? mEffS + jets[3]->PT : 0; For this manual we however preferred to use the easier to read 

version using if that most users should be familiar with. 
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} 

If 


(validThirdJet) { 

if(nSignalJets >= 2 && rEff2 > 0.2 && raEffincl > 1000.) 
countSignalEvent("AL"); 

if(nSignalJets >= 2 && rEffHT > 15. && mEffincl > 1600.) 
countSignalEvent("AM"); 

if(nSignalJets >= 3 && rEffS > 0.3 && mEffincl > 1800.) 
countSignalEvent("BM"); 

if(nSignalJets >= 3 && rEff3 > 0.4 && mEffincl > 2200.) 
countSignalEvent("BT"); 


(validMultiJet) -C 
if(nSignalJets >= 4 && rEff4 
countSignalEvent("CM"); 
if(nSignalJets >= 4 && rEff4 
countSignalEvent("CT"); 
if(nSignalJets >= 5 && rEffS 
countSignalEvent("D"); 
if(nSignalJets >= 6 && rEffS 
countSignalEvent("EL"); 
if(nSignalJets >= 6 && rEffS 
countSignalEvent("EM"); 
if(nSignalJets >= 6 && rEffS 
countSignalEvent("ET"); 


0.25 && mEffincl > 1200.) 
0.25 && mEffincl > 2200.) 
0.2 && mEffincl > 1600.) 
0.15 M mEffincl > 1000.) 
0.2 && mEffincl > 1200.) 
0.25 && mEffincl > 1500.) 


With these lines, we are actually done. The full code is listed in Fig. 

CheckMATE then needs to be re-compiled with the following commands: autoreconf; . / configure {parameters}; 
make within the CheckMATE main folder should compile the analysis and make it usable in the usual CheckMATE 
binary. 
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void Atlas_conf_2013_047x::analyze{) { 
iiiissingET->addMuons (muonsCorabined) ; 

electronsLoose = filterPhaseSpace(electroiisLoose, 10., -2.47, 2.47); 
muonsCombined = filterPhaseSpace(muonsCombined, 10., -2.4, 2.4); 
jets = filterPhaseSpaceCjets, 20., -2.8, 2.8); 
photons = filterPhaseSpace(photons, 130., -2.47, 2.47, true); 

jets = overlapRemovalCjets, electronsLoose, 0.2); 
electronsLoose = overlapRemovaKelectronsLoose, jets, 0.4); 
muonsCombined = overlapRemoval(muonsCombined, jets, 0.4); 

if C Iphotons . emptyO || ! electronsLoose.empty() II ImuonsCombined.emptyO) 

if(missingET->PT < 160.0) 

if Cjets.sizeO < 2 || jets[0]->PT <130 II jets[l]->PT < 60) 

if (fabs(jets[0]->P4().DeltaPhi{missingET->P4())) < 0.4) 

if( fabs(jets[1]->P4().DeltaPhi(missingET->P4())) < 0.4) 
return; 

std::vector<Jet*> hardjets = filterPhaseSpace(jets, 40., -2.8, 2.8); 

bool validThirdJet = (hardjets.size() < 3 II fabs(hardjets[2]->P4().DeltaPhi(missingET->P4())) > 0.4); 

bool validMultiJet = validThirdJet; 

for (int j “ 3; j < hardjets. size(); j++) 

if (fabs(hardjets[j]->P4().DeltaPhi(missingET->P4())) < 0.2) 
validMultiJet = false; 

> 


double HT = 0.; 

for(int j “ 0; j < hardjets. size() ; j+-*-) 

HT += hardjets[j]->PT; 
double mEffincl = HT + missingET->PT; 

double mEff2 = missingET->PT + jets[0]->PT + jets[l]->PT; 
double rEff2 = missingET->PT/mEff2; 

double mEff3 = 0; 
if (jets.sizeO >= 3) 

mEff3 = mEff2 + jets[2]->PT; 
double rEff3 = 0; 
if (jets.sizeO >- 3) 

rEff3 = missingET->PT/mEff3; 

double mEff4 = 0; 
if (jets.sizeO >- 4) 

mEff4 = niEff3 + jets[3]->PT; 
double rEff4 = 0; 
if (jets.sizeO >- 4) 

rEff4 = missingET->PT/mEff4; 


double mEffS = 0; 
if (jets.sizeO >- 5) 

mEffS = niEff4 + jets[4]->PT; 
double rEffS = 0; 
if (jets.sizeO >= 5) 

rEffS = missingET->PT/mEff5; 

double mEff6 = 0; 
if (jets.sizeO >- 6) 

mEff6 = mEffS + jets[5]->PT; 
double rEff6 = 0; 
if (jets.sizeO >= 6) 

rEff6 = missingET->PT/mEff6; 
double rEffHT = missingET->PT/sqrt(HT); 


> 


int nSignalJets = filterPhaseSpace(jets, 60., -2.8, 2.8).size(); 
if (validThirdJet) { 

if(nSignalJets >= 2 kk rEff2 > 0.2 kk mEffincl > 1000.) 
countSignalEventC'AL") ; 

if(nSignalJets >= 2 kk rEffHT > 15. && mEffincl > 1600.) 
countSignalEventC'AM") ; 

if(nSignalJets >= 3 kk rEff3 > 0.3 kk mEffincl > 1800.) 
countSignalEventC'BM") ; 

if(nSignalJets >= 3 kk rEff3 > 0.4 kk mEffincl > 2200.) 
countSignalEventC'BT") ; 


if (validMultiJet) { 

if(nSignalJets >= 4 kk rEff4 
countSignalEventC'CM") ; 
if(nSignalJets >= 4 kk rEff4 
countSignalEventC'CT") ; 
if(nSignalJets >= 5 kk rEffS 
countSignalEventC'D") ; 
if(nSignalJets >= 6 kk rEff6 
countSignalEventC'EL") ; 
if(nSignalJets >= 6 kk rEff6 
countSignalEventC'EM") ; 
if(nSignalJets >= 6 kk rEff6 
countSignalEventC'ET") ; 


0.25 && mEffincl > 1200.) 
0.25 kk mEffincl > 2200.) 
0.2 && mEffincl > 1600.) 
0.15 kk mEffincl > 1000.) 
0.2 && mEffincl > 1200.) 
0.25 kk mEffincl > 1500.) 


FIG. 1: Full source code of the simplified reimplementation of atlas_conf _2013_047 into CheckMATE. 


B. Example 2: Recasting a more complicated analysis 

The previous example was a rather simple search that did not contain many complicated features within the actual 
analysis code. In this example we want to consider another analysis with 3 leptons and large missing transverse 
momentum in the final state, atlas_1402_7029, see Ref. m- This analysis requires us to define specific isolation 
conditions for leptons, tag b and r jets and use advanced kinematic variables. We will also implement some outflow 
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steps to help track the signal efficiency when each cut is applied in turn. 


Detector Setting in AnalysisManager 

We run the AnalysisManager as usual and add all the necessary information, including the data for all 24 signal 
regions (we do not go into more detail here: This step is identical to the analysis before). When it comes to the 
detector simulation part, things become more involved now. The given analysis requires two specific electron isolation 
conditions: the first one needs the scalar sum of transverse momenta of tracks with px > 0.4 in the vicinity of 
Ai? < 0.3 to the candidate to be at most 16 % of the transverse momentum of the electron. 

[...] 

3. Settings for Detector Simulation 
3.1: Miscellaneous 

To which experiment does the cinalysis correspond? [(A)TLAS, (C)MS] 

A 

3.2: Electron Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 

y 

Isolation 1: 

Which objects should be considered for isolation? [(t)racks, (c)alo objects?] 
t 

What is the minimum pt of a surrounding object to be used for isolation? [in GeV] 

0.4 

What is the dR used for isolation? 

0.3 

Is there an absolute or a relative upper limit for the surrounding pt? [(a)bsolute, (r)elative] 
r 

What is the maximum pt ratio used for isolation? 

0.16 

A second condition checks the surrounding calorimeter cells in the same region to contain at most 18 % of the 
candidate’s momentum. The of the considered cells is set to be 0.1 so that random cell noise is ignored. 

Do you need more isolation criteria? [(y)es, (n)o] 

y 

Isolation 2: 

Which objects should be considered for isolation? [(t)racks, (c)alo objects?] 
c 

What is the minimum pt of a surrounding object to be used for isolation? [in GeV] 

0.1 

What is the dR used for isolation? 

0.3 

Is there cin absolute or a relative upper limit for the surrounding pt? [(a)bsolute, (r)elative] 
r 

What is the maximum pt ratio used for isolation? 

0.18 

That is all for the electrons. For the muons we also have a condition which is very similar to the first electron condition 
but with different numbers: 

3.3: Muon Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 

y 

Isolation 1: 

Which objects should be considered for isolation? [(t)racks, (c)alo objects?] 
t 

What is the minimum pt of a surrounding object to be used for isolation? [in GeV] 

1.0 

What is the dR used for isolation? 

0.3 

Is there an absolute or a relative upper limit for the surrounding pt? [(a)bsolute, (r)elative] 
r 

What is the maximum pt ratio used for isolation? 

0.12 

Do you need more isolation criteria? [(y)es, (n)o] 
n 

Photons are not needed in our analysis: 

3.4: Photon Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 
n 
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The analysis demands vetoes against jets that originated from 6-quarks. For that, a 6-tagging algorithm is used that 
is tuned to a working point signal efficiency of 80 %. Furthermore, ‘medium’ r-tags are used to identify signal events. 
We therefore have to include these objects in the AnalysisManager: 

3.5: Jets 

Which dR cone radius do you Wcint to use for the FastJet algorithm? 

0.4 

What is the minimum pt of a jet? [in GeV] 

20.0 

Do you need a separate, extra type of jet? [(y)es, (n)o] 
n 

Do you want to use b-tagging? [(y)es, (n)o] 

y 

b-Tagging 1: 

What is the signal efficiency to tag a b-jet? [in %] 

80 

Do you need more b tags? [(y)es, (n)o] 
n 

Do you want to use tau-tagging? [(y)es, (n)o] 

y 

Note that when r-tagging is activated, all three working points ‘loose’, ‘medium’ and ‘tight’ are automatically tagged 
and available in the analyses. This step will finish the AnalysisManager part and create the analysis skeleton file as 
usual. 


Analysis: Setup 

With all the parameters set, we can continue with the analysis codej^ Since we also want to check the cutflow 
of our analysis, we should book cutflow regions to make sure the _cutflow.dat file is produced at the end of a 
CheckMATE analysis run. For this we need to examine the initializeO function in our analysis skeleton file. Here 
all the signal regions we defined above are already booked: 

I bookSignalRegions("SR0taua01;SR0taua02;SR0taua03;SR0taua04;SR0taua05;... 

For outflows we add a similar line with all the desired cutflow region names. In this tutorial, let us check the number 
of events we lose after the trigger, after the event selection and how many we get with 0, 1 or 2 r’s: 

I bookCutflowRegions("CR0_A11;CRl_Trigger;CR2_Selection;CR3_0Tau;CR3_lTau;CR3_2Tau;") 

Note that regions are automatically sorted alphabetically. We therefore constructed the names such that they will 
appear in logical order in the output file. 


Analysis: Event Cleaning 


As before, we focus on the analyze () function in the analysis source file. We start with the definition of the 
interesting final state objects and the overlap removals, whose proper definitions can be found in m- Note that 
here we have to consider both ‘medium’ and ‘tight’ electrons, where the former are used for the event cleaning at the 
beginning and the latter are necessary for the signal region tests later. We also have to remove electrons that lie in 
the vicinity of each other, i.e. if two electrons are closer than AR = 0.1, the one with lower energy should be removed. 
This is done using the same overlapRemovalO function as before, but now with only one particle vector parameter 
given 

void Atlas_1402_7029::analyze() { 

missingET->addMuons(muonsCombined); 

electronsMedium = filterPhaseSpaceCelectronsMedium, 10., -2.47, 2.47, false); 
electronsTight = filterPhaseSpace(electronsTight, 10., -2.47, 2.47, false); 
muonsCombined = filterPhaseSpaceCmuonsCombined, 10., -2.4, 2.4); 
jets = filterPhaseSpaceCjets, 20., -2.5, 2.5); 

electronsMedium = overlapRemovaKelectronsMedium, 0.1); 
electronsTight = overlapRemoval(electronsTight, 0.1); 
jets = overlapRemoval(jets, electronsMedium, 0.2); 
electronsMedium = overlapRemovaKelectronsMedium, jets, 0.4); 
electronsTight = overlapRemoval(electronsTight, jets, 0.4); 
muonsCombined = overlapRemoval(muonsCombined, jets, 0.4); 


It should be noted that the implementation of the analysis we describe here slightly differs from the public version used in CheckMATE, 
as the public version considers more cutfiows which sometimes require a different ordering. 

Note that overlapRemovaKelectronsMedium, electronsMedium, 0.1) must not be used: It will always return an empty list as each 
electron in the first vector finds an ‘overlapping’ AR = 0 electron in the second vector, namely itself. 
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Next follows an analysis-specific ‘resonance removal’ which forces leptons that form opposite-sign same-flavour pairs 
with an invariant mass smaller than 12 GeV to be removed. We can do this using a simple double-loop. For electrons, 
we only need to test signal electrons (‘tight’) but they must not form an invariant mass pair with ‘medium’ electrons 
either. Since ‘tight’ electrons are a subset of ‘medium’ electrons this will also remove any pairs formed by two ‘tight’ 
electrons. 

std: :vector<ElGctron.*> noResonanceElecs; 
for (int t = 0; t < electronsTight.sizeO; t++) { 
bool valid = true; 

for (int m = 0; m < electronsMedium.sizeO; m++) { 

if (elGctronsMediumCm]->ChargG*electronsTight[t]->ChargG > 0) 
continue; 

if ( (electronsMediumCm]->P4() + electronsTight[t]->P4()).M() < 12.) 
valid = false; 

} 

if (valid) 

noResonanceElecs.push_back(electronsTight[t]); 

} 

We do the same for muons and store the cleaned vectors: 

Std::vector<Muon*> noResonanceMuons; 
for (int t = 0; t < muonsCombined.size(); t++) { 
bool valid = true; 

for (int m = 0; m < muonsCombined.size(); m++) { 

if (muonsCombined[m]->Charge*muonsCombined[t]->Charge > 0) 
continue; 

if ( (muonsCombined[m]->P4() + muonsCombined[t]->P4()).M() < 12.) 
valid = false; 

} 

if (valid) 

noResonanceMuons.push_back(muonsCombined[t]); 

} 

electronsTight = noResonanceElecs; 
muonsCombined = noResonanceMuons; 

We further select our signal final state objects by applying our defined isolation conditions. Note that applying the 
function without any arguments will apply all stored conditions on the respective object, i.e. both conditions for the 
electron and the single condition for the muon: 

I electronsTight = filterlsolation(electronsTight); 

I muonsCombined = filterlsolation(muonsCombined); 

Let us now keep track of all events we start with in the outflow: 

I countCutflowEvent("CR0_A11"); 


Analysis: Event Trigger 

The next step tests if the event could have passed at least one of the applied trigger conditions. For the present 
analysis, various triggers with different momentum thresholds for the considered objects are tested. We do not 
consider any turn-on curves but simply apply a 100 % trigger efficiency if there are objects that pass the respective 
criteria: 

bool trigger = false; 

if( electronsTight.sizeO > 0 && electronsTight[0]->PT > 25.) 
trigger = true; 

else if( muonsCombined.sizeO > 0 && muonsCombined[0]->PT > 25.) 
trigger = true; 

else if( electronsTight.sizeO > 1 && electronsTight[0]->PT > 14. kk electronsTight[1]->PT > 14.) 
trigger = true; 

else if( electronsTight.sizeO > 1 kk electronsTight[0]->PT > 25. kk electronsTight[1]->PT > 10.) 
trigger = true; 

else if( muonsCombined.sizeO > 1 kk muonsCombined[0]->PT > 14. kk muonsCombined[1]->PT > 14.) 
trigger = true; 

else if( muonsCombined.sizeO > 1 kk muonsCombined[0]->PT > 18. kk muonsCombined[1]->PT > 10.) 
trigger = true; 

else if( electronsTight.sizeO > 0 && muonsCombined.size() > 0 && electronsTight[0]->PT > 18. kk muonsCombined[0]->PT > 10.) 
trigger = true; 

else if( electronsTight.sizeO > 0 && muonsCombined.size() > 0 && electronsTight[0]->PT > 10. kk muonsCombined[0]->PT > 18.) 
trigger = true; 

if( !trigger ) 
return; 

Our cutfiow should tell us how many events pass this stage: 

I countCutflowEvent("CRl_Trigger"); 
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Analysis: Event Selection 


The present three lepton analysis considers signal regions with tan leptons and for that purpose we have to consider 
the jets that have been tagged as “medium” tau jets by using the checkTauTag function. It is important to note that 
tagged jets a priori do not have to pass conditions on the number and charge of reconstructed tracks. In the analysis 
we have to explicitly ensure this by testing the charge of the tested jets. Events then have to contain exactly three 
signal leptons (e, /i, r) but not exactly three r only: 


std::vector<Jet*> tauJets; 

for(int j = 0; j < jets.sizeO; j++) { 

if( checkTauTag(jets[j], "medium") kk fabs(jets[j]->Charge) == 1) 
tauJets.push_back(jets [j]); 

} 

if( ( electronsTight.size0 + muonsCombined.sizeO + tauJets.size() ) != 3 ) 
return; 

if( electronsTight.size0 + muonsCombined.sizeO == 0 ) 
return; 


Furthermore no &-jet should be present in the hnal state. We can test this by applying the checkBTagO function on 
a given jet candidate and, in our analysis, veto the event if any of these tests return true: 

for (int i = 0; i < jets.sizeO; i++) { 
if( checkBTagCjets[i])) 
return; 


Again we want to know how many events pass all of the above cuts: 

I countCutflowEvent("CR2_Selection"); 


Analysis: Signal Region Categorisation 


We now start categorising the event into the different signal regions. From this point on we will often consider 
both electrons and muons on the same footing. In certain cases the same treatment will also be applied to taus. It 
would therefore be convenient if we combined all the three leptons into a common vector which we can use if the 
kinematical cut is flavour independent. Unfortunately, the three objects are described by different C++ classes and 
hence cannot be combined trivially. However, within the analysis code there is a new class FinalStateObject into 
which electrons, muons, jets and even the missing momentum vector can be transformed. We therefore define a vector 
of FinalStateObject objects and choose to fill it with the three different kinds of lepton we have in the final state. 

Std: :vector<FinalStateObject*> leptons; 

for(int e = 0; e < electronsTight.size(); e++) { 

FinalStateObject* lep = newFinalStateObject(electronsTight[e]); 
leptons.push_back(lep); 

} 

for(int m = 0; m < muonsCombined.sizeO; m++) { 

FinalStateObject* lep = newFinalStateObject(muonsCombined[m]); 
leptons.push_back(lep); 

} 

for(int t = 0; t < tauJets.size(); t++) { 

FinalStateObject* lep = newFinalStateObject(tauJets[t]); 
leptons.push_back(lep); 

} 

Note that this ordering ensures that the tau objects always come last. With this vector we can easily test the further 
condition that all leptons have to be separated by at least AR > 0.3, without distinguishing electrons, muons or 
tau-jets: 

if (leptons [0]->P4() .DeltaRdeptons [1]->P4()) < 0.3) 
return; 

if (leptons [0]->P4() .DeltaRdeptons [2]->P4()) < 0.3) 
return; 

if (leptons [1]->P4() .DeltaRdeptons [2]->P4()) < 0.3) 
return; 


The main difference between the signal region definitions is the number of tau jets in the final state. We will use our 
outflow regions to check how many events in general have 0, 1 or 2 taus. Let us start with the definition of the two 
signal regions involving two tau jets: 

switch(tauJets.size0) { 
case 2: { 

countCutflowEvent("CR3_2tau"); 
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The first one of these 
mT2 variable 


SR2taua, requires at least 50 GeV missing energy in the event. Furthermore, it tests the 


see 


III B for more information. This variable requires two TLorentzVectors arguments, i.e. two four- 


momenta, and the mass of the invisible particle that is typically assumed to be 0, to reject SM background due to 
leptonic W decays. Signal region SR2taua then requires the maximum mT2 out of all combinations of the three signal 
leptons to be at least 100 GeV. With our above universal leptons vector, this is a very easy exercise: 


double mT2_l = iiiT2(leptons [0]->P4(), leptons [1]->P4(), 0.) 
double mT2_2 = ]nT2(leptons [0]->P4() , leptons [2]->P4(), 0.) 
double mT2_3 = mT2(leptons[1]->P4(), leptons [2]->P4(), 0.) 
double mT2max = std::max(std::max(mT2_l, niT2_2), mT2_3); 
if (missingET->PT >50. && mT2max > 100) 
countSignalEvent("SR2taua"); 


The other signal region, SR2taub, requires different total missing energy, an opposite sign tau pair with their scalar 
Pt sum to be larger than 110 GeV and their invariant mass to be between 70 GeV and 120 GeV: 

double mtautau = (tauJets[0]->P4()+tauJets[1]->P4()).M(); 
double sumpt = tauJets[0]->PT + tauJets[1]->PT; 

if (missingET->PT >60. && tauJets[0]->Charge*tauJets[1]->Charge < 0 && sumpt > 110. && mtautau >70. && mtautau < 120.) 
countSignalEvent("SR2taub"); 


We continue with the It signal region: 

breeik; 

} 

case 1: { 

countCutflowEvent("CR3_ltau"); 


Here we have to find the r and light lepton invariant mass combination that lies closest to the Higgs boson mass 
and the invariant mass of the light lepton pair. Since the tau jet is always the third object in the leptons vector — 
caused by the order we filled it above — we know which combinations to test. The invariant mass is easily calculated 
using the TLorentzVector internal M() function: 

double mTL_l = (leptons [0]->P4() + leptons [2]->P4()).M(); 

double mTL_2 = (leptons[1]->P4() + leptons[2]->P4()).M(); 

double mTL = fabs(mTL_l - 125.) < fabs(mTL_2 - 125.) ? mTL_l : mTL_2; 

double mLL = (leptons[0]->P4() + leptons[1]->P4()).M(); 


The light leptons must have same charge and the event is vetoed if an electron pair reconstruct an invariant mass 
compatible with the Z boson: 

I bool samesignl = (leptons [0]->Chcirge*leptons [1]->Charge > 0); 

I bool zElectronVeto = (leptons[0]->Type == "electron" && leptons[1]->Type == "electron" && mLL > 81.2 && mLL < 101.2); 

Lastly, these combine with requirements on the missing transverse momentum and the transverse momenta of the 
light leptons: 

double sumpt = leptons [0]->PT + leptons[1]->PT; 

if (missingET->PT >50. && leptons [0]->PT > 30. && leptons [1]->PT > 30. && sumpt >70. && mTL < 120. && samesignl icSc ! zElectronVeto) 
countSignalEvent("SRltau"); 
break; 

} 

case 0: { 

countCutflowEvent("CR3_0tau"); 


For this signal region, we first have to check if there is a same-flavour opposite-sign (SFOS) pair. 

bool sfos = false; 

double msfos = -lElO; 

if (leptons[0]->Charge * leptons[1]->Charge < 0 && leptons[0]->Type == leptons[1]->Type) 
sfos = true; 

else if (leptons[0]->Charge * leptons[2]->Charge < 0 && leptons [0]->Type == leptons [2]->Type) 
sfos = true; 

else if (leptons[2]->Charge * leptons[1]->Charge < 0 && leptons [2]->Type == leptons [1]->Type) 
sfos = true; 


Let us now continue with the test of SROtaub which requires no SFOS pair. In that case, we have to require that 
there is at least one lepton of different flavour (OF) and that the product of all charges is negative. We then have to 
find the OF combination which yields the smallest relative polar angle A$. Note that our construction has produced 
a leptons vector of either of the four possibilities eee, ee/r, epfi or fipp so one does not have to test all combinations 
to find OF pairs: 
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if (!sfos && leptons[0]->Type != leptons [2]->Type && fabs(leptons[0]->Charge + leptons[1]->Charge + leptons[2]->Charge) == 1) { 
double deltaPhil = fabs(leptons[0]->P4().DeltaPhi(leptons [2]->P4())); 
double deltciPhi2 = 0; 

if (leptons[0]->Type != leptons[1]->Type) 

deltaPhi2 = fabs(leptons[0]->P4().DeltaPhi(leptons[1]->P4())); 

else 

deltaPhi2 = fabs(leptons[1]->P4().DeltaPhi(leptons[2]->P4())); 
double mindeltaPhi = std::min(deltaPhil, deltaPhi2); 

if (inissingET->PT > 50. && leptons [0]->PT > 20. && leptons [1]->PT >20. && leptons [2]->PT > 20. && mindeltaPhi <= 1.0) 
countSignalEventC'SROtaub"); 

> 

For the SFOS signal region we have to Hnd the SFOS combination with invariant mass closest to the Z boson. We 
need this invariant mass and the transverse mass of the remaining lepton: 

else if(sfos) { 

double msfos = -lElO; 
double mTthird = 0; 

if (leptons [0]->Charge * leptons [1]->Charge < 0 && leptons [0]->Type == leptons [1]->Type) -C 
double minv = (leptons[0]->P4() + leptons[1]->P4()).M() 
if (fabs(minv - 92.) < fabs(msfos - 92.)) { 
msfos = minv; 

mtThird = mTdeptons [2]->P4() , missingET->P4()) ; 

} 

} 

if (leptons [0]->Charge * leptons [2]->Charge < 0 && leptons [0]->Type == leptons [2]->Type) -C 
double minv = (leptons[0]->P4() + leptons [2]->P4()).M() 
if (fabs(minv - 92.) < fabs(msfos - 92.)) { 
msfos = minv; 

mtThird = mTdeptons [1]->P4() , missingET->P4()) ; 

} 

} 

else if (leptons[2]->Charge * leptons[1]->Charge < 0 && leptons[2]->Type == leptons[1]->Type){ 
double minv = (leptons[2]->P4() + leptons[1]->P4()).M() 
if (fabs (minv - 92.) < fabs (msfos - 92.)) {. 
msfos = minv; 

mtThird = mTdeptons [0]->P4(), missingET->P4()); 

} 

} 

Lastly, some signal regions specifically veto if the three lepton invariant mass lies within the Z boson mass window: 

I double m31 = (leptons[0]->P4() + leptons[1]->P4() + leptons[2]->P4()).M(); 

The analysis ends with a binning of missing energy, msfos and mtThird 

if (12. < msfos && msfos < 40) { 

if (0. <= mtThird && mtThird < 80) { 

if (50. <= missingET->PT && missingET->PT < 90.) 

countSignalEvent("SROtauaOl"); 
else if (90. <= missingET->PT) 

countSignalEvent("SR0taua02"); 

} 

else if (80. <= mtThird ) ■( 


Analysis Testing 


To check the implementation of our analysis, let us run CheckMATE on a simple model that is also analysed 
within m- a simplihed supersymmetric scenario with direct production of the lightest chargino and the second 
lightest neutralino, both with mass m.-± = m^o = 625 GeV. These both decay into the lightest neutralino with mass 
TO^o = 375 GeV and two leptons via an intermediate slepton with mass = 500 GeV. By charge conservation 
this will lead to an odd number of charged leptons in the final state and is consequently a good candidate for our 
underlying three lepton study. We take the corresponding slha spectrum file [SU [33] and use Prospino [iOl liT] to 
calculate the production cross section to be cr « 2.893 pb. We then use Herwig++ [42] to generate events and analyse 
these with CheckMATE using the following input parameter card: 

## General Options 

[Mandatory Parameters] 

Name: My_Trilepton_Run 
Analyses: atlas_1402_7029X 

## Process Information (Each new process ’X’ must start with [X]) 

[Simplified] 

XSect: 2.893 FB 
XSectErr: 0 FB 

Events: /hdd/data/validation/trileptons/exclusion_A/mClN2_625_0_mNl_375_0.hepmc 
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Looking at the /emalysis/ sub-folder within the results directory reveals the desired 
000_atlas_1402_7029X_cutflow.dat and 000_atlas_1402_7029X_signal.dat hies we booked in our analysis 
code. The cuthow hie tells us the number of events remaining after each selection step in our analysij^ 


# ATLAS 

# 3 leptons 

# tutorial 


Inputfile: 

XSect: 

Error: 

MCEvents: 
SmnOfWeights: 
SumOfWeights2: 
NormEvents: 


/tutorial/results/My_Trilepton_Run/delphes/000_delphes.root 
2.893 fb 
0 fb 
10000 
10000 
10000 
58.7279 


Cut 

Sum_W 

Sum_W2 

Acc 

N_Norr[i 

CR0_A11 

10000 

10000 

1 

58.7279 

CRl_Trigger 

7448 

7448 

0.7448 

43.7405 

CR2_Selection 

1647 

1647 

0.1647 

9.67249 

CR3_0Tau 

1306 

1306 

0.1306 

7.66986 

CR3_lTau 

272 

272 

0.0272 

1.5974 

CR3_2Tau 

69 

69 

0.0069 

0.405223 


Furthermore we can check the numbers for each signal region: 


# ATLAS 

# 3 leptons 

# tutorial 


Inputfile: 


/hdd/Scindbox/trileptonupdate/results/My_Trilepton_Run/delphes/000_delphes.root 


XSect: 


2.893 fb 



Error: 


0 fb 



MCEvents: 


10000 



SmnOfWeights: 
SrnnOfWeights2: 
NormEvents: 

10000 

10000 

58.7279 



SR 

Smn_W 

Smn_W2 

Acc 

N_Norm 

SROtauaOl 

1 

1 

0.0001 

0.00587279 

SR0taua02 

4 

4 

0.0004 

0.0234912 

SR0taua03 

[...] 

SROtaub 

4 

4 

0.0004 

0.0234912 

4 

4 

0.0004 

0.0234912 

SRltau 

20 

20 

0.002 

0.117456 

SR2taua 

38 

38 

0.0038 

0.223166 

SR2taub 

11 

11 

0.0011 

0.0646007 

Finally, CheckMATE tells us Result: Allowed, 


suit for r: r_max = 0.36209. One might be surprised if one 
compares to Fig. 7a in m in which the exact same point lies very close to the exclusion line. However, the discrepancy 
is to be expected as CheckMATE only considers the most sensitive signal region whereas within sensitivities of 
various signal regions are combined using correlations to which CheckMATE does not have access. It is therefore 
reasonable that CheckMATE returns a weaker limit which is however still close to the nominal limit. 


To finish this subsection, we would like to add several general remarks on the validation procedure. Firstly, in most 
cases the results are sensitive to the actual Monte Carlo generator used. This is especially true if the signal regions 
probe events that depend on the additional jet radiation from the initial state. The user should make sure to use 
the same Monte Carlo and settings as those reported by the collaborations. Secondly, one should keep in mind that 
on many occasions the outflow is given for a specihc subprocess, e.g. only some of the physically allowed decays are 
actually included in the produced sample. Another possible issue is the use of truth level/generator cuts, e.g. in the 
outflow sample only events with at least one truth lepton of pt > 10 GeV are included. Since these settings are not 
always straightforward to implement, it can be convenient to start the comparison of outflows at some later stage 
rather that at the initial ’raw’ events. Similarly, users may also wish to produce a subset of events actually analysed 
by the experiment, e.g. only include leptonic W events in a di-lepton analysis. This has an advantage of reducing 
computation time and increasing statistical accuracy. 

From a practical point of view, trivial coding errors can be spotted in the outflow by noting that for some of the steps 
there is no change in the efficiency or the change is much larger than the one reported by the experiment. Generally, 
even when the numbers between CheckMATE and the experiment are not in perfect agreement, the general trends in 
the outflow should remain similar. Still, one should keep in mind that Delphes is only a rough parametrisation of that 


Note that if you do the same, you will find slightly different numbers due to finite Monte Carlo statistics. 
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detector response and differences of 10%-20% occasionally can occur. Last but not least, if the discrepancy persist, 
it is worth to directly contact the conveners responsible for the analysis that the user is trying to implement. It may 
well turn out that some details of the simulation are not explicitly mentioned in the experimental report. 


C. Example 3: Inventing a completely new analysis 


So far, we have discussed the implementation of current experimental collider studies. In this section, we want to 
implement a completely new analysis, a monojet search at 14 TeV with an integrated luminosity of 100 fb“^. The 
monojet analysis at 14 TeV is based on Ref. [43] but with additional signal regions accommodating harder transverse 
momentum cuts on the leading jet and stricter cuts on missing transverse energy. 

We run the AnalysisManager and enter the general information as previously described. We call our analysis: 

I Analysis Name: 

atlas_monojet_at_14_tev 


We define five signal regions: 

2. Information on Signal Regions 

List all signal regions (one per line, finish with cin empty line): 

Ml 

M2 

M3 

M4 

M5 

We now have to provide the background numbers for each signal region. However, these numbers are evidently not 
known at this stage, so we choose no. 

I Is the SM expectation B known? [(y)es, (n)o]? 
n 


The AnalysisManager will display the following message: 

Signal regions are registered but without einy numbers associated to them. 

IMPORTANT: The analysis will be created and can then be used like any other 

analysis. CheckMATE will skip the model exclusion tests as long as 
the expectation is not known. You can e.g. use CheckMATE on background 
samples to estimate B and dB. As soon as you know these numbers, 
run the AnalysisManager again and use the (e)dit feature to add them. 

Press key to continue! 


We will simulate the dominant Standard Model backgrounds and enter the background numbers for all signal regions 
later, because for that we have to write the analysis first. In the next step we have to define all detector level objects 
as discussed in the previous examples. 

3. Settings for Detector Simulation 
3.1: Miscellaneous 

To which experiment does the enalysis correspond? [(A)TLAS, (C)MS] 

A 

3.2: Electron Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 
n 

3.3: Muon Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 

y 

Isolation 1: 

Which objects should be considered for isolation? [(t)racks, (c)alo objects?] 
t 

What is the minimum pt of a surrounding object to be used for isolation? [in GeV] 

1.0 

What is the dR used for isolation? 

0.2 

Is there cin absolute or a relative upper limit for the surrounding pt? [(a)bsolute, (r)elative] 
a 

What is the maximum surrounding pt used for isolation [in GeV]? 

1.8 


Do you need more isolation criteria? [(y)es, (n)o] 
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3.4: Photon Isolation 

Do you need any particular isolation criterion? [(y)es, (n)o] 
n 

3.5: Jets 

Which dR cone radius do you want to use for the FastJet algorithm? 
0.4 

What is the minimum pt of a jet? [in GeV] 

20.0 

Do you need a separate, extra type of jet? [(y)es, (n)o] 
n 

Do you want to use b-tagging? [(y)es, (n)o] 
n 

Do you want to use tau-tagging? [(y)es, (n)o] 
n 


After answering the remaining questions of the AnalysisManager, the source and header hies are generated. Every¬ 
thing is now set by the AnalysisManager and we start to write the analysis code. 

We want to select events satisfying the following trigger condition and kinematic cuts. First, we apply an online 
trigger condition with 80 GeV and assume a flat trigger efficiency of 100%. All events are required to have 
> 150 GeV and the leading jet must satisfy pt > 150 GeV and \r]\ < 2.8. Events with isolated electrons (muons) 
with Pt > 20 {pr > 10 GeV) are rejected. We also veto events if there are more than three jets with pT > 30 GeV 
and \r]\ < 2.8. In order to reduce the QGD background, we require an azimuthal separation between the jets and the 
missing transverse momentum vector: A())(jet, p™®®) > 0.4. Finally, we choose the following set of cuts to dehne our 
signal regions. 

• Ml: $T> 220 GeV and pi^'^dingjet ^ 280 GeV 

• M2: :$T> 340 GeV and ^ 

• M3: $T> 450 GeV and p^^^ingjet ^ 

• M4: $T> 500 GeV and p^^^ingjet ^ 

• M5: $T> 550 GeV and p^^^^ingjet ^ 


We now have all the information to write the analysis code. Since we have already discussed two examples, we just 
present the full analysis code without discussing it in detail. 

#include "atlas_monojet_at_14_tGv.h" 

void Atlas_monojGt_at_14_tGv::initialize() { 

SGtAnalysisName("atlas_monojGt_at_14tGv"); 

SGtInf ormationC " 

"0#targGts gGUGric ATLAS monojGt studiGs\n" 

’'{3#14 TeV, 100/fb\n" 

SGtLuminosity(100.0*units::INVFB); 

ignorGC'towGrs"); // ThGSG won’t road towGr or track information from thG 
ignorG("tracks"); // DalphGS output branchGS to savG computing tima. 
bookSignalRegions("Ml;M2;M3;M4;M5;"); 

} 
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void Atlas_nionojet_at_14_tev: :ciiialyze0 ■[ 
missingET->addMuons (inuonsCombinGd); 

electronsMedium = filterPhaseSpaceCelectronsMedium, 20., -2.47, 2.47); 
muonsCombined = filtGrPhaseSpaceCmuonsCombined, 10., -2.4, 2.4); 
jets = filterPhaseSpaceCjets, 20., -2.8, 2.8); 

jets = overlapRemoval(jets, electronsMedium, 0.2); 
electronsMedium = overlapRemoval(electronsMedium, jets, 0.4); 
muonsCombined = overlapRemoval(muonsCombined, jets, 0.4); 

std::vector<Muon*> isoMuons = filterlsolation(muonsCombined); 

//trigger 

if (missingET->P4().Et() < 80.) 
return; 

if (isoMuons.size0 > 0 || electronsMedium.sizeO > 0 ) 
return; 

if ( jets.sizeO > 3 && jets[3]->PT > 30. ) 
return; 

for ( int i = 0; i < jets.sizeO && jets[i]->PT > 30.; i++) ■{ 
if ( fabs(missingET->P4().DeltaPhi(jets[i]->P4())) < 0.4) 
return; 

} 

if ( jets.sizeO == 0 I | jets[0]->PT < 150. ) 
return; 

if (missingET->P4() .EtO < 150.) 
return; 

if (jets[0]->PT > 280.) { 

if (missingET->P4() .EtO > 220.) 
countSignalEventC'Ml") ; 

} 

if (jets[0]->PT > 340.) { 

if (missingET->P4() .EtO > 340.) 
countSignalEvent("M2"); 

} 

if (jets[0]->PT > 450.) { 

if (missingET->P4() .EtO > 450.) 
countSignalEvent("M3"); 

} 

if (jets[0]->PT > 500.) { 

if (missingET->P4() .EtO > 500.) 
countSignalEvent("M4"); 

} 

if (jets[0]->PT > 550.) { 

if (missingET->P4() .EtO > 550.) 
countSignalEvent("M5"); 

} 


void Atlas_monojet_at_14_tev::finalizeO { 

} 

After having implemented the monojet analysis code, we have to determine the expected numbers of Standard Model 
background events for all signal regions. Firstly, we briefly discuss the major backgrounds to the monojet signal. 
The vv) + j is the dominant irreducible background. In general, VF(—/ Iv) + j is a non-negligible background 
even though it contains one charged lepton (£ = , /i^). However, if the charged lepton is not reconstructed, it 
will have similar final states as the monojet signature. The hadronic r decays oi W + j production also yield a 
sizeable contribution to the background of the monojet signal. The ti background happens to be relatively small 
and consequently we will omit it for simplicity as well as the other sub-dominant backgrounds. We have simulated 
the Z j and W j samples with the MC event generator Sherpa |44j using leading order matrix elements for up 
to 3 partons and using massive h/c quarks with the CTEQIO parton distribution functions [45] . Our background 
estimate is given in Table |Tj Now we have to provide our background numbers for each signal region from Table |T] to 
CheckMATE. We again call the AnalysisManager but this time choose the (e)dit analysis information option. 
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Signal Region 

Ml 

M2 

M3 

M4 

M5 

SM prediction 

733900 ± 4860 

197501 ± 2592 

51776 ± 1231 

18461 ± 640 

7477 ± 364 


TABLE I: Number of expected background events for all signal regions. Only statistical errors are included. All 

numbers are normalized to 100 fb“^ at y/s = 14 TeV. 


What would you like to do? 

-(l)ist all analyses, 

-(a)dd a new analysis to CheckMATE, 

-(e)dit analysis information, 

-(r)emove an analysis from CheckMATE] 
e 

You can now edit or update some of the information for a given ainalysis. 
Enter the identifier of the analysis you want to edit: 
atlas_monojet_at_14_tev 

Analysis atlas_monojet_at_14_tev successfully read in. 

Do you want to... 

...list all defined entries? (1) 

...restart the detector settings questions? (d) 

...enter signal region numbers for observation cind background? (n) 


We choose n and enter the number of observed events (which for a hypothetical analysis will be the sam^^ as the 
expected background), expected background events and the error of the background events for each signal region. 
The AnalysisManager will then calculate the model independent upper limits. 

We are now in the position to perform studies with our new analysis and as an example we choose to apply our 
implementation to a simple supersymmetric scenario. We consider direct production of scalar top quark (stop) pairs 
in association with one jet |46| . We assume that the stops decay into a charm quark and the lightest neutralino with a 
branching ratio of 100%. If the mass splitting between the stop and the neutralino is very small, the charm jets cannot 
be reconstructed and hence our signal is a single highly energetic jet recoiling against missing momentum. We choose 
a benchmark point with mi_^ = 400 GeV and m^o = 350 GeV. We calculate the cross section with Prospino [H] and 
obtain a = 2.15 pb. The events are generated as a matched sample pp —>■ tit\ + j with Madgraph_aMC@NLO gTl and 
the showering is done with Pythia |48]. Analysing our resulting event file with CheckMATE, we obtain a r-value ^ of 
0.32 for Ml which is determined to be the most sensitive signal region. Consequently we conclude that the parameter 
point is expected to be allowed in this analysis after 100 fb“^ at LHC-14. 


D. Auxiliary Information 


The following functionalities might also be useful for the user in certain scenarios, which is why we list them 
separately here without reference to a particular example: 


File Booking 

The CheckMATE internal files _signal.dat are automatically put into the globally chosen output directory with 
individual prefixes for each analysis and each separate input file. Sometimes a user might want to store extra infor¬ 
mation that is determined during the analysis, as an example kinematical distributions in order to draw histograms, 
in separate files. CheckMATE provides simple to use functions that create these files in the same manner as the 
_signal.dat files, i.e. into the user-chosen output directory with individual prefixes. 

To do this, the user should use the bookFile (filename, noHeader) function within the initialize () part of the 
analysis code (similarly to bookSignalRegions): this will create a file with the correct prefix before filencune in the 
CheckMATE output directory. If one wants to create a text file and wants general analysis information to automatically 
be written to the top of that file, noHeader should be set to false, otherwise true. The function returns an integer 
number, which should always be stored by the user in a variable, for example int iFile = bookFileC'hist .txt", 
false);. 

Then, fNames [iFile] can be used to get the absolute path to the file, which can e.g. be used to book histograms 
via Root (see Root documentation for more information on Root histogramming). 


We implicitly assume the null hypothesis here, ie. that no signal above the SM expectation is present. 
The r-value is defined as the number of expected signal events divided by the expected 95% C.L. value. 
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Alternatively, one can write text information directly into the file by using f Streams [iFile] for example as follows: 
f Streams [iFile] << "I have a jet with pt " << jet[0]->PT << std::endl;. If iFile is defined as a member 
of the analysis class, the above code can be used in all three functions initialize!), ainalyzeO , finalize!). 


Random Numbers 

If random numbers have to be used, rand!)/!RAND_MAX+1.) should be used to get a uniformly distributed random 
number between 0 and I. The reason to use raind!) and not e.g. Root based random number generators is that 
CheckMATE has a parameter to fix the random seed of a run. This fixed seed only applies for the rand!) function. 
As a fixed seed should lead to completely deterministic results, one should avoid using any other random number 
generator. 


V. SUMMARY 

We have introduced the AnalysisManager of CheckMATE that allows for the easy implementation of new analyses, 
either invented by user or following experimental searches. The program guides the user through the numerous 
choices possible for final state objects, simplifying many of the complicated LHC definitions. In order to aid the 
actual coding of analyses, many useful, routinely performed functions are included. Additionally, a suite of common 
LHC kinematical variables are also implemented. Finally, all the statistical variables already available in CheckMATE 
can be calculated automatically. 

CheckMATE can be downloaded from: 

http://checkmate.hepforge.org/ 


Detailed program documentation can be found at: 

http://checkmate.hepforge.org/documentation/index.html 
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